Using Pedagogical Advisement in Technology-Based Environments

Fifty-eight Education graduate students took a forty-minute technology-based instructional module on introductory statistics with a built-in solicited guidance mechanism. Subjects were randomly assigned to programs that used one of four types of advisement: on-screen digitized video of a human advisor, on-screen text-based advisor, pull-down digitized video of a human advisor, or pull-down text-based advisor. Results indicated that The on-screen video-based advisor condition resulted in higher advisor use than both the textbased and video-based pull-down advisor conditions. Advisor use was significantly correlated with performance during instruction, to time spent during instruction, and to television hours watched per week, but not with retention scores. Two nonsignificant, but inviting, findings were that the video-based on-screen advisors were used twice as much as text-based on-screen advisors and active learners used advisement three times as often as passive learners. One of the more interesting practical questions involving the use of the Internet or multimedia for instructional purposes concerns the best way to “coach” a student who needs additional help in understanding complex concepts. It is suggested by some researchers (e.g., Smith, 1992) that this coaching (or advisement) acts as an intermediate point between generative and supplantive instruction. This study investigated the use of video pedagogical advisor to assist students in learning statistical concepts via computer-based instruction. The use of advisement in computer-mediated lessons has been well supported since microcomputers first came into use. Studies by Tennyson and his associates (Johansen & Tennyson, 1983; Tennyson, 1980; 1981; Tennyson & Buttrey, 1980) found that designing computer-based lessons using learner control with advisement increased performance when compared to lessons designed with either adaptive (program) control or learner control without advisement. In most cases, performance increase was accompanied by a decrease in total instructional time compared to adaptive control methods (Gray, 1988; Tennyson, 1980, 1981). Although some advisement strategies have been shown to be effective, getting students to use advisement has been problematic. Gay and her colleagues, for example, presented data that a text-based, on-line advisor was not used very often (Gay & Mazur, 1993; Gay, Trumbull, & Mazur, 1991). In the sense that an advisor “cues” the learner regarding important issues during instruction, the placement and ease of use of the advisor is an important concern, particularly with more passive learners (Lee & Lehman, 1993). The limited number of programs that employ an advisor or coach frequently present the information in text. With more recent technological advances, use of audio and video advisors have become viable alternatives to text. Research concerning the use of video and audio advisors, however, has been limited or has had limited generalizability (see, for example, Austin, 1994). Even so, a video advisor in the form of a “talking head” has been applied quite cleverly in certain computer-based training programs. For example, Zelos’ Shoot Video Like a Pro provides talking head “experts” to provide learners with tips to improve their emerging skills. The experts in this software are built into the camera-like interface, are both male and female, and come from a variety of ethnic backgrounds. Programs using video coaches fit well into a model for human-computer interaction proposed by Streitz (1988). In Streitz’s model the learner is confronted with an interaction problem that requires the learner, as a user, to build a representation of the tutoring system. In addition to the “learner” and the “system,” the model proposes a problem mediator and a human tutor. The human tutor reflects the fact that a person often does not learn strictly on his or her own initiative but because a person (boss, teacher, friend, etc.) proposes a learning path. According to Streitz, this person functions as a problem mediator who makes suggestions or asks questions about specific content domains. From another perspective, the video advisor functions as what the filmmaker Luis Bunuel refers to as the “explicador” (Bunuel, 1984; Plowman, 1994). During the early period of the cinema in Bunuel’s native Saragossa, explicadors were used to explain the action of the movie to the audience and guide inexperienced moviegoers from scene to scene. In our time, the explicador is confined to a rectangular image of the medium close-up television image. Video advisors are a metaphor we can understand and accept. Television has trained us to do so. This understanding is what semioticians such as Solomon (1988) refer to as our “communal sense.” The purpose of this study was to explore the use of advisement and the modality of the advisement mechanism in a technology-based module. Independent variables were the placement of the advisor (pull-down menu access to the advisor vs. an on-screen access) and the modality of the advisor (digitized video of a human advisor vs. a text-based advisor). Dependent variables were the number of times the learner chose to use the advisor, delayed posttest scores, performance on practice items during instruction, and motivation indices based on Keller’s ARCS model. Students’ scores on the Computer Anxiety Ratings Scale (Gerbing & Anderson, 1988; Heinssen, Glass, & Knight; 1987; Miller & Rainer, 1995) were used as a covariate. Time spent in instruction and learning style based on Lee & Lehman’s Passive Active Learning Scale, or PALS (19??) were collected for planned post-hoc comparisons. Our expectations were: • the on-screen conditions would result in more frequent advisor use than the pull-down conditions. • the video advisor would be used more frequently than the text advisor. • the use of an advisor would lead to higher motivation indices in general and video advisement in particular would result in higher motivation indices than text. • using an advisor would be positively correlated with performance during instruction and on the posttest. • learners who were classified as “passive” on the PALS instrument would use advisement less than those classified as “active” in the pulldown conditions, but that this difference would decrease or be eliminated in the on-screen conditions. Method Subjects were 43 females and 15 males aged 21 through 57 years, with an average age of 35. Subjects were drawn from three graduate educational research survey courses and one graduate psychological principles of learning course at a southeastern university. Subjects had attended an average of 3.2 computer-related classes, had 4.9 years of computer experience, and watched television 11.3 hours per week. Subjects were randomly assigned to one of the four experimental conditions. Materials and Instruments A forty-minute instructional module on statistics was developed using Macromedia Authorware. The lesson covered sampling distributions, hypothesis testing, and type I and II errors. The instruction employed a “ruleexample-practice” format, and incorporated color, graphics, text, sound, and feedback. Based on the subject’s assignment, advisement for each of the 25 practice items was available either as video or text and could be accessed either on-screen or via a pull-down menu. The program was reviewed for content accuracy by two experienced educational researchers and was formatively evaluated . The computer program tracked student performance, time, and advisor use during instruction. An 18-item delayed posttest was constructed based on the content objectives for the statistical module. A 36-item motivation instrument was a modification of Keller’s Instructional Motivational Scale (Keller, 1987). This scale includes statements related to attention, relevance, confidence, and satisfaction specifically oriented toward instructional computer programs. Computer anxiety was measured by the 7-item Computer Anxiety Rating Scale (CARS) developed by Marc Miller and R. Kelly Rainer, Jr., from the original 19-item scale developed by Heinssen, Glass, and Knight (1987). This scale has been shown to reliably measure high and low anxiety constructs, with Cronbach alpha scores of .82 and .73 respectively. Passive/Active learning style was measured by the 31-item Likert type Passive Active Learning Scale (PALS) developed by Ben Lee and James Lehman. This scale has been shown to reliably measure the passive/active learning construct, with a Cronbach alpha score of .81. Procedure A variety of demographic data including age, gender, computer classes attended, computer experience, TV hours watched per week, and graduate classes taken were collected prior to the instruction. At the same time, subjects completed the CARS anxiety measure. Subjects were randomly assigned to one of two levels of each independent variable: on-screen video, pull-down video, on-screen text, and pull-down text. Immediately after subjects completed the instructional module, they completed the ARCS-based attitude-toward-instruction instrument. Exactly one week later, the PALS instrument and the posttest were administered to the subjects prior to in-class study of the material covered in the instructional module. Results A one-way ANOVA indicated significant differences in advisor use between groups (F=3.385, p = .025). The video-based on-screen advisor condition resulted in higher advisor use (EMM=5.25) than both the text-based (EMM=.866) and video-based (EMM=1) pull-down advisor conditions, but not the text-based on-screen condition, although it was used on average almost twice as much (EMM=2.384). A one-way ANCOVA indicated significant differences in advisor use between groups when controlling for anxiety (F=3.352, p = .026). Results were similar to the ANOVA, in which the video-based on-screen advisor condition resulted in higher advisor use (EMM=5.26) than both the text-based (EMM=.862) and video-based (EMM=.949) pull-down advisor conditions, but not the text-based on-screen condition (EMM=2.431). One-way ANOVAs and ANCOVAs failed to yeild significant differences between groups for any other measures, including PALS scores. Pearson Product Moment Correlations indicated that advisor use was significantly related to performance during instruction (.407, p<.01) to time spent during instruction (.432, p < .01), and to television hours watched per week (.292, p < .05). Anxiety and computer classes were significantly related (.341, p < .05), as were performance during instruction and time spent during instruction (.427, p < .01) and performance during instruction and motivation (.462, p < .01). Performance during instruction was significantly related to posttest scores (.402, p < .01) and to motivation (.27, p< .05). Television hours watched was significantly related to computer experience (.283, p < .05). PALS scores were not significantly related to advisor use. Correlations performed between advisor use and PALS scores by condition indicated no signficant relationship. A similar analysis of advisor use and television hours watched indicated that the signficant correlation found earlier was located specifically in the text obvious condition. Because advisor use was postively skewed and bi-modal, the variable was also converted to a categorical variable and analyzed via chi-square analysis. Thirty-five scores of zero were recoded as “none” for advisor use. The remaining 23 fell between 1 and 7, and 12 and 17. These scores were recoded as “some” advisor use. A chisquare analysis was then run with condition and the new advisor use variable. The analysis indicated a significant deviation from expected values (p = .002). Analysis of adjusted standardized residuals indicated that the text-based pull-down advisor and video-based on-screen advisor condtions differed from the expected. Discussion Our first expectation, that the on-screen conditions would result in more frequent advisor use than the pulldown conditions, was partially supported. Subjects who had on-screen access to an advisor used advisement more than four times as often on average than those who had pull-down access to an advisor (3.965 vs. .931). Figure 1 shows data that illustrates partial support for the main effect of advisor placement. The on-screen video condition was statistically different from both pulldown conditions. The on-screen text condition, although higher than both pulldown conditions, was not statistically different. This data may indicate that the modality bias against a text-based advisor which Gay, et al., found can be overcome by placement of the advisor.